Method and apparatus for multi-exposure photography, and storage medium

ABSTRACT

A method for multi-exposure photography includes: receiving a screen touch operation, and determining a collected frame corresponding to the screen touch operation, a subject area image corresponding to the screen touch operation, and an alignment parameter of the subject area image corresponding to the screen touch operation; acquiring a currently collected frame in real time; synthesizing the subject area image corresponding to the screen touch operation, or the subject area image corresponding to the screen touch operation together with subject area images corresponding to all historical screen touch operations into the currently collected frame according to corresponding alignment parameters, taking the synthesized image to be an image with a multi-exposure processed effect and displaying the image with the multi-exposure processed effect; and taking an image with a multi-exposure processed effect generated after a final screen touch operation to be a multi-exposed result image.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Chinese patent application No.201910923409.3 filed on Sep. 27, 2019, the disclosure of which is herebyincorporated by reference in its entirety.

BACKGROUND

Typically, the multi-exposure function of a digital camera is in factimplemented through a process of synthesizing multiple pictures into onepicture in a certain way in real time. For example, a human body profilecan be synthesized with other background to reach a special artisticeffect. FIG. 1 illustrates a synthesized picture subjected tomulti-exposure processing in a first application scenario.Alternatively, pictures to be synthesized contain a same background butmultiple subjects associated with a same person, and the positions ofthe subjects in the background are different among the pictures, toreach an interesting effect of a synthesized picture containing multipledoppelgangers of the same person.

SUMMARY

The present disclosure generally relates to the technology ofmulti-exposure photography, and more specifically to a method andapparatus for multi-exposure photography, and a storage medium.

According to a first aspect of embodiments of the disclosure, a methodfor multi-exposure photography is provided, including:

receiving a screen touch operation;

determining a collected frame corresponding to the screen touchoperation;

determining, according to a first rule, a subject area imagecorresponding to the screen touch operation from the collected framecorresponding to the screen touch operation; determining, according to asecond rule, an alignment parameter of the subject area imagecorresponding to the screen touch operation;

acquiring a currently collected frame in real time;

when determining that the screen touch operation is the first receivedscreen touch operation, synthesizing, according to the alignmentparameter of the subject area image corresponding to the screen touchoperation, the subject area image corresponding to the screen touchoperation into the currently collected frame, taking the synthesizedimage to be an image with a multi-exposure processed effect anddisplaying the image with the multi-exposure processed effect; or whendetermining that the screen touch operation is not the first receivedscreen touch operation, synthesizing, according to correspondingalignment parameters, the subject area image corresponding to the screentouch operation and subject area images corresponding to all historicalscreen touch operations into the currently collected frame, taking thesynthesized image to be the image with the multi-exposure processedeffect and displaying the image with the multi-exposure processedeffect; and

determining an image with a multi-exposure processed effect from theimages with the multi-exposure processed effect, which are generatedafter a final screen touch operation, to be a multi-exposed resultimage.

According to a second aspect of embodiments of the disclosure, anapparatus for multi-exposure photography is provided, including:

a processor; and

a memory, configured to store instructions executable by the processor,

wherein the processor is configured to:

receive a screen touch operation;

determine a collected frame corresponding to the screen touch operation;

determine, according to a first rule, a subject area image correspondingto the screen touch operation from the collected frame corresponding tothe screen touch operation;

determine, according to a second rule, an alignment parameter of thesubject area image corresponding to the screen touch operation;

acquire a currently collected frame in real time;

when determining that the screen touch operation is the first receivedscreen touch operation, synthesize, according to the alignment parameterof the subject area image corresponding to the screen touch operation,the subject area image corresponding to the screen touch operation intothe currently collected frame, take the synthesized image to be an imagewith a multi-exposure processed effect and display the image with themulti-exposure processed effect; or when determining that the screentouch operation is not the first received screen touch operation,synthesize, according to corresponding alignment parameters, the subjectarea image corresponding to the screen touch operation and subject areaimages corresponding to all historical screen touch operations into thecurrently collected frame, take the synthesized image to be the imagewith the multi-exposure processed effect and display the image with themulti-exposure processed effect; and

determine an image with a multi-exposure processed effect from theimages with the multi-exposure processed effect, which are generatedafter a final screen touch operation, to be a multi-exposed resultimage.

According to a third aspect of embodiments of the disclosure, anon-transitory computer-readable storage medium is provided, whereininstructions in the storage medium, when executed by a processor of amobile terminal, enable the mobile terminal to execute a method formulti-exposure photography, comprising:

receiving a screen touch operation;

determining a collected frame corresponding to the screen touchoperation;

determining, according to a first rule, a subject area imagecorresponding to the screen touch operation from the collected framecorresponding to the screen touch operation;

determining, according to a second rule, an alignment parameter of thesubject area image corresponding to the screen touch operation;

acquiring a currently collected frame in real time;

when determining that the screen touch operation is the first receivedscreen touch operation, synthesizing, according to the alignmentparameter of the subject area image corresponding to the screen touchoperation, the subject area image corresponding to the screen touchoperation into the currently collected frame, taking the synthesizedimage to be an image with a multi-exposure processed effect anddisplaying the image with the multi-exposure processed effect; or whendetermining that the screen touch operation is not the first receivedscreen touch operation, synthesizing, according to correspondingalignment parameters, the subject area image corresponding to the screentouch operation and subject area images corresponding to all historicalscreen touch operations into the currently collected frame, taking thesynthesized image to be the image with the multi-exposure processedeffect and displaying the image with the multi-exposure processedeffect; and

determining an image with a multi-exposure processed effect from theimages with the multi-exposure processed effect, which are generatedafter a final screen touch operation, to be a multi-exposed resultimage.

It should be understood that the general description above and detaileddescription below are merely exemplary and explanatory, and are notintended to restrict the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings referred to in the specification are a part ofthis disclosure, and provide illustrative embodiments consistent withthe disclosure and, together with the detailed description, serve toillustrate some embodiments of the disclosure.

FIG. 1 illustrates a synthesized picture subjected to multi-exposureprocessing in a first application scenario.

FIG. 2 illustrates a flowchart of a method for multi-exposurephotography according to some embodiments of the disclosure.

FIG. 3 illustrates a flowchart of the process of determining a subjectarea image in response to a screen touch operation and according to anobject image segmentation approach in a method for multi-exposurephotography according to some embodiments of the disclosure.

FIG. 4 illustrates a detailed flowchart of a method for multi-exposurephotography according to some embodiments of the disclosure.

FIG. 5A illustrates a schematic diagram of a collected framecorresponding to a first received screen touch operation, and a picturedisplayed on a screen at the moment of the screen touch operation,according to the method for multi-exposure photography illustrated inFIG. 4.

FIG. 5B illustrates a schematic diagram of a collected framecorresponding to a second received screen touch operation, and a picturedisplayed on the screen at the moment of the screen touch operation,according to the method for multi-exposure photography illustrated inFIG. 4.

FIG. 5C illustrates a schematic diagram of a collected framecorresponding to a third received screen touch operation, and a picturedisplayed on the screen at the moment of the screen touch operation,according to the method for multi-exposure photography illustrated inFIG. 4.

FIG. 5D illustrates a multi-exposed result picture generated accordingto the method for multi-exposure photography illustrated in FIG. 4.

FIG. 6A illustrates a multi-exposed result picture generated accordingto the method for multi-exposure photography illustrated in FIG. 4.

FIG. 6B illustrates another multi-exposed result picture generatedaccording to the method for multi-exposure photography illustrated inFIG. 4.

FIG. 7 illustrates a block diagram of an apparatus for multi-exposurephotography according to some embodiments of the disclosure.

DETAILED DESCRIPTION

Exemplary embodiments (examples of which are illustrated in theaccompanying drawings) are elaborated below. The following descriptionrefers to the accompanying drawings, in which identical or similarelements in two drawings are denoted by identical reference numeralsunless indicated otherwise. The exemplary implementation modes may takeon multiple forms, and should not be taken as being limited to examplesillustrated herein. Instead, by providing such implementation modes,embodiments herein may become more comprehensive and complete, andcomprehensive concept of the exemplary implementation modes may bedelivered to those skilled in the art. Implementations set forth in thefollowing exemplary embodiments do not represent all implementations inaccordance with the subject disclosure. Rather, they are merely examplesof the apparatus and method in accordance with certain aspects herein asrecited in the accompanying claims.

Synthesis processing may be implemented by simply adding the pictures ina fixed proportion. Specifically, multiple pictures are photographedconsecutively, with the photographed multiple pictures containing thesame background; during preview, the multiple pictures which have notbeen aligned are simply synthesized averagely for display; when thephotography of all the pictures is completed, all of the pictures arealigned in an off-line manner; the area where a picture subject islocated is determined according to a difference between pictures, andthe subjects are extracted separately to obtain a final result, i.e.,obtaining a multi-exposed photography picture.

It can be seen from the description of the process above that, in therelated art, during the preview of multi-exposure photography, averagesynthesization of multiple pictures cannot be performed until thephotography of all the pictures is completed.

FIG. 2 illustrates a flowchart of a method for multi-exposurephotography according to some embodiments of the disclosure. The methodmay be applied to a terminal. As illustrated in FIG. 2, the methodincludes the following blocks.

In block S21, a screen touch operation is received, a collected framecorresponding to the screen touch operation is determined, a subjectarea image corresponding to the screen touch operation is determinedfrom the collected frame corresponding to the screen touch operationaccording to a first rule, and an alignment parameter of the subjectarea image corresponding to the screen touch operation is determinedaccording to a second rule.

In block S22, a currently collected frame is acquired in real time.

In block S23, when determining that the screen touch operation is thefirst received screen touch operation, the subject area imagecorresponding to the screen touch operation is synthesized into thecurrently collected frame according to the alignment parameter of thesubject area image corresponding to the screen touch operation, thesynthesized image is taken to be an image with a multi-exposureprocessed effect, and the image with the multi-exposure processed effectis displayed. When determining that the screen touch operation is notthe first received screen touch operation, the subject area imagecorresponding to the screen touch operation and subject area imagescorresponding to all historical screen touch operations are synthesizedinto the currently collected frame according to corresponding alignmentparameters, the synthesized image is taken to be the image with themulti-exposure processed effect, and the image with the multi-exposureprocessed effect is displayed.

In block S24, an image with a multi-exposure processed effect isdetermined to be a multi-exposed result image from the images with themulti-exposure processed effect, which are generated after a finalscreen touch operation.

In the above block S21, the screen touch operation may include variousoperations. For example, the screen touch operation may be a screenclicking operation, a screen long-press operation, an operation ofsliding on a screen or the like. When the screen touch operation is asliding operation, the sliding track may be regular or irregular. Thesliding track may include any open and/or closed curve. The curve hereinincludes a straight line, and a broken line with more than onebreakpoint.

In this embodiment, the recognized subject area image may be a human, ananimal, a plant, a still article or the like. Since a recognitionoperation will be conducted each time a screen touch operation isreceived, multiple subject area images may be recognized. When multiplesubject area images are recognized, the subject area image recognizedeach time may be associated with a same subject. For example, thesubject area image recognized each time may be associated with the samehuman. The subject area image recognized each time may be associatedwith different subjects as well. For example, the subject area imagerecognized at a first time is associated with a human, and the subjectarea image recognized at a second time is associated with a stillarticle, etc.

In the above block S23, when determining that the screen touch operationis not the first received screen touch operation, the subject area imagecorresponding to the screen touch operation and subject area imagescorresponding to all historical screen touch operations are synthesizedinto the currently collected frame according to corresponding alignmentparameters. The subject area image corresponding to the screen touchoperation may be synthesized into the currently collected frameaccording to the alignment parameter of the subject area imagecorresponding to the screen touch operation determined in block S21. Thesubject area images corresponding to all historical screen touchoperations prior to receiving the screen touch operation may besynthesized into the currently collected frame successively according tothe alignment parameters of the subject area images corresponding to therespective historical screen touch operations. During the synthesisoperation, the subject area image corresponding to the screen touchoperation and the subject area images corresponding to all thehistorical screen touch operations are synthesized into a same collectedframe.

In the above block S24, the multi-exposed result image may be determinedin many ways. By way of example, when determining that themulti-exposure photography has ended, the subject area imagecorresponding to the final screen touch operation and the subject areaimages corresponding to all the historical screen touch operations maybe synthesized into the collected frame acquired at the moment ofreceiving the final screen touch operation, and the synthesized imageobtained at this time is taken to be the multi-exposed result image.When determining that the multi-exposure photography ends, the subjectarea image corresponding to the final screen touch operation and thesubject area images corresponding to all the historical screen touchoperations may also be synthesized into the collected frame acquired atthe moment of determining that the multi-exposure photography ends, andthe synthesized image obtained at this time is taken to be themulti-exposed result image.

It can be seen from the above embodiment that, in the technical solutionaccording to embodiments of the disclosure, each time a screen touchoperation is received, an operation of recognizing a subject area imageand an operation of synthesizing the recognized subject area image willbe triggered. However, in conventional multi-exposure photography, allthe subjects involved in synthesis processing are extracted fromphotographed pictures (referred to hereinafter as original pictures)respectively, that is to say, each extracted subject corresponds anoriginal picture. Hence, a collected frame corresponding to a screentouch operation herein is equivalent to an original picture inmulti-exposure photography, and the screen touch operation is equivalentto an operation of triggering the photography of the original picture inthe multi-exposure photography. The subject area image recognizedaccording to the screen touch operation is equivalent to a subjectextracted from the current original picture. All the subject area imagesrecognized according to previously received screen touch operations areequivalent to subjects extracted from previously photographed originalpictures. It can be seen therefrom that, in this embodiment, the imagewith the multi-exposure processed effect is equivalent to a real-timepreview picture generated each time an original picture formulti-exposure is photographed. That is to say, during multi-exposurephotography, the multi-exposure processed effect of a currentlyphotographed original picture and previously photographed originalpictures can be previewed in real time each time an original picture isphotographed, without the need of conducting an average synthesisoperation to obtain a multi-exposed effect picture after the photographyof all the pictures is completed. It can be seen that the technicalsolution of this embodiment provides a user with the function ofpreviewing a multi-exposure processed effect in real time moreconveniently and more intuitively. The multi-exposed result imageobtained at the end of multi-exposure photography operations is one ofthe images with the multi-exposure processed effect previewed in realtime; that is, the photography effect of “what you see is what you get”is achieved, improving user experience. Moreover, in the technicalsolution of this embodiment, the means of triggering photography by ascreen touch operation is totally different from the means of adding apicture by pressing a shutter in the related art, and triggeringphotography by a screen touch operation is more convenient for a user'sphotography operation and improves user experience.

Another method for multi-exposure photography is further provided inthis embodiment. The first rule involved in the method may include:determining the subject area image corresponding to the screen touchoperation according to at least one of the screen touch operation or anobject image segmentation approach.

It can be seen from the above description that the approach to recognizea subject area image in an original picture in the technical solution ofthis embodiment may at least include the following three approaches.

A subject area image corresponding to a screen touch operation isdetermined according to the screen touch operation. That is to say, animage area corresponding to the screen touch operation initiated by auser can be recognized as the subject area image. For example, when thescreen touch operation initiated by the user includes a slidingoperation, the image area corresponding to the screen touch operationmay be determined according to a sliding track, and this image area maybe extracted as the subject area image. According to this approach, thescreen touch operation initiated by the user is considered as main basisfor the recognition operation, so that the user's requirement ofautonomously selecting a subject to be selected can be better satisfied,improving user experience.

The subject area image corresponding to the screen touch operation isdetermined according to an object image segmentation approach. That isto say, an object image segmentation approach is used to automaticallyrecognize the subject area image from the collected frame correspondingto the screen touch operation, equivalent to recognizing the subjectarea image from the original picture of which the photography istriggered by the screen touch operation. The object image segmentationapproach is a commonly used object segmentation technique, which willnot be described in detail. In this approach, the function ofautomatically recognizing a subject is realized based on the objectimage segmentation approach, simplifying the flow of recognition, andimproving the efficiency of recognition.

The subject area image corresponding to the screen touch operation isdetermined according to the screen touch operation and the object imagesegmentation approach. That is to say, the screen touch operationinitiated by the user and the object image segmentation approach arecombined to serve as a basis for the recognition operation. In thisapproach, during the recognition of the subject, the advantages of theobject image segmentation approach and the user autonomously selecting asubject to be synthesized are combined, so that the recognition of thesubject area image is more accurate and better fits user requirementswhile ensuring the efficiency of recognition.

Another method for multi-exposure photography is further provided inthis embodiment. In the method, the process of determining a subjectarea image corresponding to a screen touch operation according to thescreen touch operation and an object image segmentation approach, asillustrated in FIG. 3, at least includes the following operations.

In block S31, an image area corresponding to the screen touch operationis determined from the collected frame corresponding to the screen touchoperation.

In the above block S31, image area corresponding to the screen touchoperation may be determined in a random way. For example, when thescreen touch operation includes a clicking operation, the coordinates,in the coordinate system of the collected frame corresponding to thescreen touch operation, of the clicking position of the clickingoperation may be acquired, which are referred to as the coordinates ofthe clicking position for short. An area of a set size which containsthe coordinates of the clicking position is determined as an image areacorresponding to the screen touch operation. The set size may bepre-configured by the system, or may be configured by the userautonomously. Furthermore, when the screen touch operation includes along-press operation, the way to determine an image area correspondingto the long-press operation is substantially the same as that ofdetermining the image area corresponding to the clicking operation,which will not be described here.

For another example, when the screen touch operation includes a slidingoperation, the coordinates, in the coordinate system of the collectedframe corresponding to the screen touch operation, of the sliding startpoint of the sliding operation are referred to as the coordinates of thesliding start point for short. The coordinates, in the coordinate systemof the collected frame corresponding to the screen touch operation, ofall the sliding points during sliding are referred to as the coordinatesof the sliding points for short. The coordinates, in the coordinatesystem of the collected frame corresponding to the screen touchoperation, of the sliding end point are referred to as the coordinatesof the sliding end point for short. When the sliding track is an opencurve, an area of a set size which contains the coordinates of theclicking start point, the coordinates of all the sliding points and thecoordinates of the sliding end point may be determined to be an imagearea corresponding to the screen touch operation. When the sliding trackis a closed curve, the coordinates of the clicking start point, thecoordinates of all the sliding points and the coordinates of the slidingend point may be taken to be a boundary to divide the collected framecorresponding to the screen touch operation into an area inside theclosed curve and an area outside the closed curve. An area of a set sizewhich contains the area inside the closed curve may be determined to bean image area corresponding to the screen touch operation. The set sizemay be pre-configured by the system, or may be configured by the userautonomously.

In block S32, position information and image information contained inthe image area are input into a deep neural network.

In the above block S32, the position information contained in the imagearea may include the coordinate range of the image area in thecoordinate system of the collected frame corresponding to the screentouch operation. The image information contained in the image area mayinclude pixel information, light/shadow information, etc. of the imagearea. The pixel information may include any one or more of hue,saturation and brightness. The light/shadow information may include allinformation having a direct effect on light rays, for example,brightness and contrast.

In block S33, the subject area image is recognized from the image areaby means of the deep neural network.

In the above block S33, any object segmentation approach may be employedin the deep neural network to recognize the subject area image, whichwill not be particularly specified.

It can be seen that, according to the technical solution of thisembodiment, an area to be recognized can be accurately positionedaccording to a user's screen touch operation, and then the subject areaimage can be automatically recognized using a deep neural network, thatis, the artificial intelligence technique. In this way, the subject areaimage can be recognized at a higher precision, thus improving theprocessing effect of multi-exposure photography, solving the problem inthe related art that terrible fault tolerance of a subject area of apicture may easily lead to misjudgment.

Another method for multi-exposure photography is also provided in thisembodiment. The second rule involved in the method may include one ofthe following:

determining, according to a pre-configured alignment parameter, thealignment parameter of the subject area image corresponding to thescreen touch operation;

conducting, according to a pre-configured alignment algorithm, alignmentcomputation on the currently collected frame acquired in real time andthe collected frame corresponding to the screen touch operation, toobtain the alignment parameter of the subject area image correspondingto the screen touch operation; or

conducting, according to the pre-configured alignment algorithm,alignment computation the currently collected frame acquired in realtime and collected frames corresponding to the historical screen touchoperations, to obtain the alignment parameter of the subject area imagecorresponding to the screen touch operation.

The operation of determining, according to a pre-configured alignmentparameter, the alignment parameter of the subject area imagecorresponding to the screen touch operation is equivalent to using thepre-configured alignment parameter to obtain a synthesized image with amulti-exposure photography processed effect based on the alignmentoperation experience. In this way, the flow of a synthesis operation canbe simplified, and the efficiency of the synthesis operation can beimproved.

Alignment computation is conducted, according to a pre-configuredalignment algorithm, on the currently collected frame acquired in realtime and the collected frame corresponding to the screen touchoperation, to obtain the alignment parameter of the subject area imagecorresponding to the screen touch operation. The applied alignmentmanner is to take the collected frame corresponding to each receivedscreen touch operation, that is, the original picture photographed eachtime a photography operation is triggered, as a reference to performalignment of a currently collected frame acquired in real time, so as toobtain an image with a multi-exposure processed effect.

Alignment computation is conducted, according to the pre-configuredalignment algorithm, on the currently collected frame acquired in realtime and collected frames corresponding to the historical screen touchoperations, to obtain the alignment parameter of the subject area imagecorresponding to the screen touch operation. The applied alignmentmanner is to take one of the frames that have been collected whenreceiving a screen touch operation, that is one of the photographedoriginal pictures as a reference to perform alignment of the currentlycollected frame acquired in real time, so as to obtain an image with amulti-exposure processed effect.

It can be seen that, in the technical solution of this embodiment, inthe process of synthesizing an image with a multi-exposure processedeffect in real time, any manner of alignment may be used to conduct asynthesis operation. As described above, in this embodiment, a subjectarea image is recognized in real time at each screen touch operation,that is, each time a photography operation is triggered, and theaccuracy of recognition is high. Therefore, with the reliability ofrecognizing a subject area image being guaranteed, the quality of animage with a multi-exposure processed effect synthesized in any mannerof alignment can be guaranteed to a certain extent. The problems of blurand distortion, and disorder of photographed subjects caused byconducting average synthesization after the photography of multiplepictures is completed in the related art can be avoided.

Another method for multi-exposure photography is provided in thisembodiment. The method also includes the following operations: receivingan editing operation initiated for the subject area image correspondingto any one or more of the historical screen touch operations; andprocessing, according to the editing operation, the correspondingsubject area image, and taking the processed image to be an updatedimage with the multi-exposure processed effect.

In the technical solution of this embodiment, by means of displaying animage with a multi-exposure processed effect in real time, a user canview the effect of multi-exposure photography timely. Further, thefunction of adjusting the image with the multi-exposure processed effectin real time to improve the effect of a final synthesized picture isprovided for the user. For example, if the effect such as the posture,expression and position of some subject area image cannot satisfy userrequirements, the subject area image can be particularly processedaccording to an editing operation initiated by the user. This operationwill not affect the other subject area images. Consequently, the userexperience is improved.

Another method for multi-exposure photography is provided in thisembodiment. In the method, an editing operation may include an operationof modifying the subject area image and/or an operation of deleting thesubject area image.

By means of the operation of modifying the subject area image, variousparameters of a specified subject area image can be modified in theimage with the multi-exposure processed effect displayed in real timeduring multi-exposure photography. The various parameters of the subjectarea image may include the image position, the image size, the imagepixel, the image brightness, the image transparency, etc.

Through the operation of deleting the subject area image, one or morespecified subject area images may be deleted from all the recognizedsubject area images during multi-exposure photography.

FIG. 4 illustrates a schematic diagram of a detailed flow of a methodfor multi-exposure photography according to some embodiments of thedisclosure. FIG. 5 illustrates schematic diagrams of collected framesand images with a multi-exposure processed effect corresponding toscreen touch operations during multi-exposure photography illustrated inFIG. 4. FIG. 6 illustrates final multi-exposed result images obtainedaccording to the method illustrated in FIG. 4.

The process of carrying out a method for multi-exposure photography isexplained in conjunction with FIG. 4 and FIG. 5. The process, asillustrated in FIG. 4, includes the following blocks.

In block S401, a multi-exposure photography flow is started, and acurrently collected frame is acquired and displayed in real time.

In this block, the currently collected frame acquired in real time, asillustrated in the left part of FIG. 5A, is an actually collectedoriginal image frame.

In block S402, a screen touch operation is received for the first time,a collected frame corresponding to the screen touch operation isdetermined, and a first subject area image is determined from thecollected frame according to the screen touch operation.

In this embodiment, the above block S402 may be carried out in thefollowing operations.

In block S402 a, a screen touch operation initiated by a user isreceived, and an image frame Ia at the current moment is collected.

As illustrated in the right part of FIG. 5A, the user initiates thescreen touch operation by clicking a photographed human in the screen.

As illustrated in the left part of FIG. 5A, the image frame Ia of thecurrent moment is collected to serve as an original frame, equivalent toa photographed original picture. The image frame Ia is the collectedframe corresponding to the screen touch operation.

In block S402 b, the image frame Ia is taken as a reference frame foralignment computation, to generate an alignment tool A.

In this block, the alignment algorithm used in the alignment computationmay be preset.

In block S402 c, an image area, corresponding to the screen touchoperation, in the image frame Ia is determined.

In this block, a first area can be acquired around the position at whichthe user performs a clicking operation. The size of the first area maybe configured by the system or the user. As illustrated in the rightpart of FIG. 5A, the first area corresponding to the user's clickingoperation is the area identified with a rectangular block in the displayscreen. The area mapped to the image frame Ia by the first area is theimage area corresponding to the screen touch operation. For simplifieddescription, the method of determining the area mapped to the imageframe Ia by the first area is explained, with the example of the displayscreen and the image frame Ia using a same coordinate system. Thecoordinate range of the first area in the coordinate system of thedisplay screen can be first determined, and the area having the samecoordinate range in the coordinate system of the image frame Ia is thenconfirmed to be the area mapped to the image frame Ia by the first area.

In block S402 d, the pixel information and the position information ofthe image area corresponding to the screen touch operation are inputinto a deep neural network; and in the deep neural network, a subjectarea image in the image area is automatically predicted and is extractedas a recognized first subject area image, which can be referred to asobject Sa for short.

In block S403, the alignment parameter of the first subject area imageis determined, and the determined alignment parameter of the firstsubject area image is used to synthesize the first subject area imageinto the currently collected frame acquired in real time, for display.

In this embodiment, the above block S403 may be carried out in thefollowing operations:

In block S403 a, the currently collected frame I acquired in real timeis aligned with the reference frame Ia by using the alignment tool Agenerated in block S402 b, to obtain an alignment parameter Wa.

In block S403 b, the object Sa, to which affine transformation has beenmade, is drawn, by using the alignment parameter Wa, into the currentlycollected frame I acquired in real time, for display.

The operation of making affine transformation to the object Sa and thendrawing the transformed object into the currently collected frame Iacquired in real time is equivalent to synthesizing the object Sa intothe currently collected frame I acquired in real time. The imagedisplayed at this time is an image with a multi-exposure processedeffect, such as the image displayed at the right part of FIG. 5B.

In block S404, a second screen touch operation is received, a collectedframe corresponding to the screen touch operation is determined, and asecond subject area image is determined from the collected frameaccording to the screen touch operation.

In this embodiment, the above block S404 may include the followingoperations:

In block S404 a, a screen touch operation initiated by a user isreceived, and an image frame Ib of the current moment is collected.

As illustrated in the left part of FIG. 5B, the image frame Ib of thecurrent moment is collected to serve as an original frame, equivalent toa photographed original picture. The image frame Ib is the collectedframe corresponding to the screen touch operation.

As illustrated in the right part of FIG. 5B, the user may initiate thescreen touch operation by clicking a photographed portrait in thescreen. Since the image with the multi-exposure processed effect isdisplayed in the screen at this time, the image contains twophotographed subjects. One of the photographed subjects is a subject tobe synthesized, that is, the subject area image determined in blockS402, namely the object Sa. The other is a real photographed subject,that is, the object Sb illustrated in the right part of FIG. 5B. Thephotographed portrait clicked by the user is the object Sb.

In block S404 b, the image area in the image frame Ib corresponding tothe screen touch operation is determined.

In this block, a second area can be acquired around the position atwhich the user performs a clicking operation. The size of the secondarea may be configured by the system or the user. As illustrated in theright part of FIG. 5B, the second area corresponding to the user'sclicking operation is the area identified by a hand-shaped cursor with arectangular block in the display screen. The area mapped to the imageframe Ib by the second area is the image area corresponding to thescreen touch operation. The principle of determining the area mapped tothe image frame Ib by the second area is the same as that of determiningthe area mapped to the image frame Ia by the first area, which will notbe described.

In block S404 c, the pixel information and the position information ofthe image area corresponding to the screen touch operation are inputinto a deep neural network; and in the deep neural network, a subjectarea image in the image area is automatically predicted and is extractedas a recognized second subject area image, which can be referred to asobject Sb for short.

In block S405, the alignment parameter of the second subject area imageis determined, the determined alignment parameter of the second subjectarea image is used to synthesize the second subject area image into thecurrently collected frame acquired in real time, and the subject areaimage corresponding to the historical screen touch operation, namely thefirst subject area image, is also synthesized into the collected frame,for display.

In this embodiment, the above block S405 may include the followingoperations.

In block S405 a, the currently collected frame I acquired in real timeis aligned with the reference frame Ia by using the alignment tool Agenerated in block S402 b, to obtain an alignment parameter Wa_b.

In block S405 b, the object Sa, to which affine transformation has beenmade, is drawn, by using the alignment parameter Wa obtained in blockS403 a, into the currently collected frame I acquired in real time.

In block S405 c, the object Sb, to which affine transformation has beenmade, is drawn, by using the alignment parameter Wa×Wa_b−1 (that is, theinverse matrix of the alignment parameter Wa_b), into the currentlycollected frame I acquired in real time, for display.

The drawing operations in blocks S405 b and S405 c are to draw theobject Sa and the object Sb into the same collected frame. The imagedisplayed at this time is an image with a multi-exposure processedeffect, such as the image displayed at the right part of FIG. 5C.

In block S406, a third screen touch operation is received, a collectedframe corresponding to the screen touch operation is determined, and athird subject area image is determined from the collected frameaccording to the screen touch operation.

In this embodiment, for the operations in the above block S406,reference can be made to the operations in the above block S404, whichwill not be described. When a screen touch operation initiated by a useris received, an image frame Ic of the current moment is collected, asillustrated in the left part of FIG. 5C. As illustrated in the rightpart of FIG. 5C, the screen touch operation initiated by the user istargeted at the subject area image, that is, the actually photographedportrait Sc. The object Sc is the third subject area image determined inthis embodiment.

In block S407, the alignment parameter of the third subject area imageis determined, the third subject area image is synthesized into thecurrently collected frame acquired in real time, and the subject areaimages corresponding to the historical screen touch operations, namelythe first subject area image and the second subject area image, are alsosynthesized into the collected frame, for display.

In this embodiment, the above block S407 may include the followingoperations:

In block S407 a, the currently collected frame I acquired in real timeis aligned with the reference frame Ia by using the alignment tool Agenerated in block S402 b, to obtain an alignment parameter Wa_c.

In block S407 b, the object Sa, to which affine transformation has beenmade, is drawn, by using the alignment parameter Wa obtained in blockS403 a, into the currently collected frame I acquired in real time.

In block S407 c, the object Sb, to which affine transformation has beenmade, is drawn, by using the alignment parameter Wa_b obtained in blockS405 a, into the currently collected frame I acquired in real time.

In block S407 d, the object Sc, to which affine transformation has beenmade, is drawn, by using the alignment parameter Wa×Wa_c−1 (that is, theinverse matrix of the alignment parameter Wa_c), into the currentlycollected frame I acquired in real time, for display.

The drawing operations in blocks S407 b, S407 c and S407 d are to drawthe object Sa, the object Sb and the object Sc into the same collectedframe. The image displayed at this time is an image with amulti-exposure processed effect.

In block S408, an instruction to end multi-exposure photography isreceived, and an image with a multi-exposure processed effect generatedafter a final screen touch operation is output, displayed and stored asa multi-exposed result image.

The image with the multi-exposure processed effect corresponding to thefinal screen touch operation may be obtained by synthesizing the subjectarea image corresponding to the final screen touch operation and thesubject area images corresponding to all the historical screen touchoperations into the collected frame acquired at the moment of receivingthe final screen touch operation. The image with the multi-exposureprocessed effect corresponding to the final screen touch operation mayalso be obtained by synthesizing the subject area image corresponding tothe final screen touch operation and the subject area imagescorresponding to all the historical screen touch operations into thecollected frame acquired at the moment of receiving the instruction toend multi-exposure photography.

In this block, when the user confirms that all the pictures have beenadded completely and clicks the complete button as illustrated in theleft part of FIG. 5C, the instruction to end multi-exposure photographyis received. The generated multi-exposed result image is as illustratedin FIG. 5D. With actually photographed pictures as an example, themulti-exposed result images generated according to the above method areas illustrated in FIGS. 6. FIG. 6A and FIG. 6B are multi-exposedpictures finally generated after photography in two scenes. It can beseen therefrom that in the same scene, the photographed subjects (thatis, the child) extracted from different pictures are synthesized intothe same picture at different positions, reaching the interesting effectof the same picture containing multiple doppelgangers of the sameperson.

In addition, during photography according to the above method formulti-exposure photography, the image with the multi-exposure processedeffect displayed in real time provides the user with the function ofpreviewing a multi-exposure processed effect in real time. Therefore,the user can modify the image with the multi-exposure processed effectin real time as required. That is to say, on the basis of the abovemethod, an editing operation initiated by the user for one or moresubject area images in the image with the multi-exposure processedeffect can also be received. The subject area image selected by the useris correspondingly processed according to the editing operation, and theprocessed image is updated to be an image with a multi-exposureprocessed effect. For example, when the received editing operation is anoperation of deleting the subject area image, the remaining subject areaimages among all the subject area images other than the deleted subjectarea image can be synthesized into the currently collected frameacquired in real time, for display. The synthesized image obtained is anupdated image with the multi-exposure processed effect.

FIG. 7 illustrates a block diagram of an apparatus for multi-exposurephotography according to some embodiments of the disclosure. Asillustrated in FIG. 7, the apparatus includes a first portion 71, asecond portion 72, a third portion 73, a fourth portion 74 and a fifthportion 75.

The first portion 71 is configured to receive a screen touch operation,determine a collected frame corresponding to the screen touch operation,determine, according to a first rule, a subject area image correspondingto the screen touch operation from the collected frame corresponding tothe screen touch operation, and determine, according to a second rule,an alignment parameter of the subject area image corresponding to thescreen touch operation.

The second portion 72 is configured to acquire a currently collectedframe in real time.

The third portion 73 is configured to: when determining that the screentouch operation is the first received screen touch operation,synthesize, according to the alignment parameter of the subject areaimage corresponding to the screen touch operation, the subject areaimage corresponding to the screen touch operation into the currentlycollected frame, take the synthesized image to be an image with amulti-exposure processed effect and display the image with themulti-exposure processed effect.

The fourth portion 74 is configured to: when determining that the screentouch operation is not the first received screen touch operation,synthesize, according to corresponding alignment parameters, the subjectarea image corresponding to the screen touch operation and subject areaimages corresponding to all historical screen touch operations into thecurrently collected frame, take the synthesized image to be the imagewith the multi-exposure processed effect and display the image with themulti-exposure processed effect.

The fifth portion 75 is configured to determine an image with amulti-exposure processed effect from the images with the multi-exposureprocessed effect, which are generated after a final screen touchoperation, to be a multi-exposed result image.

Another apparatus for multi-exposure photography is provided in thisembodiment. The first rule includes: determining the subject area imagecorresponding to the screen touch operation according to at least one ofthe screen touch operation or an object image segmentation approach.

Another apparatus for multi-exposure photography is provided in thisembodiment. The first portion 71 may include a first sub-portion, asecond sub-portion and a third sub-portion.

The first sub-portion is configured to determine an image areacorresponding to the screen touch operation from the collected framecorresponding to the screen touch operation.

The second sub-portion is configured to input position information andimage information contained in the image area into a deep neuralnetwork.

The third sub-portion is configured to recognize the subject area imagefrom the image area by means of the deep neural network.

Another apparatus for multi-exposure photography is provided in thisembodiment. The second rule may include one of the following:determining, according to a pre-configured alignment parameter, thealignment parameter of the subject area image corresponding to thescreen touch operation; conducting, according to a pre-configuredalignment algorithm, alignment computation on the currently collectedframe acquired in real time and the collected frame corresponding to thescreen touch operation, to obtain the alignment parameter of the subjectarea image corresponding to the screen touch operation; or conducting,according to the pre-configured alignment algorithm, alignmentcomputation on the currently collected frame acquired in real time andcollected frames corresponding to the historical screen touchoperations, to obtain the alignment parameter of the subject area imagecorresponding to the screen touch operation.

Another apparatus for multi-exposure photography is provided in thisembodiment. The apparatus may further include a sixth portion and aseventh portion.

The sixth portion is configured to receive an editing operationinitiated for the subject area image corresponding to any one or more ofthe historical screen touch operations.

The seventh portion is configured to process, according to the editingoperation, the corresponding subject area image, take the processedimage to be an updated image with the multi-exposure processed effectand display the updated image with the multi-exposure processed effect.

Another apparatus for multi-exposure photography is provided in thisembodiment. An editing operation involved in the apparatus may includean operation of modifying the subject area image and/or an operation ofdeleting the subject area image.

With regard to the apparatus in the above embodiments, the specific wayfor the various portions to execute operations has been described indetail in the embodiments regarding the method, which will not bedescribed in detail here.

In some embodiments of the disclosure, an apparatus for multi-exposurephotography is provided, including a processor and a memory configuredto store instructions executable by the processor. The processor isconfigured to:

receive a screen touch operation, determine a collected framecorresponding to the screen touch operation, determine, according to afirst rule, a subject area image corresponding to the screen touchoperation from the collected frame corresponding to the screen touchoperation, and determine, according to a second rule, an alignmentparameter of the subject area image corresponding to the screen touchoperation;

acquire a currently collected frame in real time;

when determining that the screen touch operation is the first receivedscreen touch operation, synthesize, according to the alignment parameterof the subject area image corresponding to the screen touch operation,the subject area image corresponding to the screen touch operation intothe currently collected frame, take the synthesized image to be an imagewith a multi-exposure processed effect and display the image with themulti-exposure processed effect; or when determining that the screentouch operation is not the first received screen touch operation,synthesize, according to corresponding alignment parameters, the subjectarea image corresponding to the screen touch operation and subject areaimages corresponding to all historical screen touch operations into thecurrently collected frame, take the synthesized image to be the imagewith the multi-exposure processed effect, and display the image with themulti-exposure processed effect; and

determine an image with a multi-exposure processed effect from theimages with the multi-exposure processed effect, which are generatedafter a final screen touch operation, to be a multi-exposed resultimage.

With regard to the apparatus in the above embodiments, the specific wayfor the various parts to execute operations has been described in detailin the embodiments regarding the method, which will not be described indetail here.

In some embodiments of the disclosure, a non-transitorycomputer-readable storage medium is provided, wherein instructions inthe storage medium, when executed by a processor of a mobile terminal,may enable the mobile terminal to execute a method for multi-exposurephotography, including:

receiving a screen touch operation, determine a collected framecorresponding to the screen touch operation, determine, according to afirst rule, a subject area image corresponding to the screen touchoperation from the collected frame corresponding to the screen touchoperation, and determine, according to a second rule, an alignmentparameter of the subject area image corresponding to the screen touchoperation;

acquiring a currently collected frame in real time;

when determining that the screen touch operation is the first receivedscreen touch operation, synthesizing, according to the alignmentparameter of the subject area image corresponding to the screen touchoperation, the subject area image corresponding to the screen touchoperation into the currently collected frame, taking the synthesizedimage to be an image with a multi-exposure processed effect anddisplaying the image with the multi-exposure processed effect; or whendetermining that the screen touch operation is not the first receivedscreen touch operation, synthesizing, according to correspondingalignment parameters, the subject area image corresponding to the screentouch operation and subject area images corresponding to all historicalscreen touch operations into the currently collected frame, taking thesynthesized image to be the image with the multi-exposure processedeffect, and displaying the image with the multi-exposure processedeffect; and

determining an image with a multi-exposure processed effect from theimages with the multi-exposure processed effect, which are generatedafter a final screen touch operation, to be a multi-exposed resultimage.

The specific way for each of the instructions in the storage medium inthe above embodiments to execute an operation has been described indetail in the embodiments regarding the method, which will not bedescribed in detail here.

Various embodiments of the present disclosure can have one or more ofthe following advantages.

During multi-exposure photography, an image synthesis operation can beconducted in real time and a multi-exposure synthesis effect can bedisplayed, according to a screen touch operation initiated by a user. Areal-time preview function can therefore be realized, which is moreconvenient, quicker and more intuitive, improves user experience.

In the present disclosure, the terms “installed,” “connected,”“coupled,” “fixed” and the like shall be understood broadly, and can beeither a fixed connection or a detachable connection, or integrated,unless otherwise explicitly defined. These terms can refer to mechanicalor electrical connections, or both. Such connections can be directconnections or indirect connections through an intermediate medium.These terms can also refer to the internal connections or theinteractions between elements. The specific meanings of the above termsin the present disclosure can be understood by those of ordinary skillin the art on a case-by-case basis.

In the description of the present disclosure, the terms “oneembodiment,” “some embodiments,” “example,” “specific example,” or “someexamples,” and the like can indicate a specific feature described inconnection with the embodiment or example, a structure, a material orfeature included in at least one embodiment or example. In the presentdisclosure, the schematic representation of the above terms is notnecessarily directed to the same embodiment or example.

Moreover, the particular features, structures, materials, orcharacteristics described can be combined in a suitable manner in anyone or more embodiments or examples. In addition, various embodiments orexamples described in the specification, as well as features of variousembodiments or examples, can be combined and reorganized.

In some embodiments, the control and/or interface software or app can beprovided in a form of a non-transitory computer-readable storage mediumhaving instructions stored thereon is further provided. For example, thenon-transitory computer-readable storage medium can be a ROM, a CD-ROM,a magnetic tape, a floppy disk, optical data storage equipment, a flashdrive such as a USB drive or an SD card, and the like.

Implementations of the subject matter and the operations described inthis disclosure can be implemented in digital electronic circuitry, orin computer software, firmware, or hardware, including the structuresdisclosed herein and their structural equivalents, or in combinations ofone or more of them. Implementations of the subject matter described inthis disclosure can be implemented as one or more computer programs,i.e., one or more portions of computer program instructions, encoded onone or more computer storage medium for execution by, or to control theoperation of, data processing apparatus.

In some embodiments, or in addition, the program instructions can beencoded on an artificially-generated propagated signal, e.g., amachine-generated electrical, optical, or electromagnetic signal, whichis generated to encode information for transmission to suitable receiverapparatus for execution by a data processing apparatus. A computerstorage medium can be, or be included in, a computer-readable storagedevice, a computer-readable storage substrate, a random or serial accessmemory array or device, or a combination of one or more of them.

Moreover, while a computer storage medium is not a propagated signal, acomputer storage medium can be a source or destination of computerprogram instructions encoded in an artificially-generated propagatedsignal. The computer storage medium can also be, or be included in, oneor more separate components or media (e.g., multiple CDs, disks, drives,or other storage devices). Accordingly, the computer storage medium canbe tangible.

The operations described in this disclosure can be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources.

The devices in this disclosure can include special purpose logiccircuitry, e.g., an FPGA (field-programmable gate array), or an ASIC(application-specific integrated circuit). The device can also include,in addition to hardware, code that creates an execution environment forthe computer program in question, e.g., code that constitutes processorfirmware, a protocol stack, a database management system, an operatingsystem, a cross-platform runtime environment, a virtual machine, or acombination of one or more of them. The devices and executionenvironment can realize various different computing modelinfrastructures, such as web services, distributed computing, and gridcomputing infrastructures.

A computer program (also known as a program, software, softwareapplication, app, script, or code) can be written in any form ofprogramming language, including compiled or interpreted languages,declarative or procedural languages, and it can be deployed in any form,including as a stand-alone program or as a portion, component,subroutine, object, or other portion suitable for use in a computingenvironment. A computer program can, but need not, correspond to a filein a file system. A program can be stored in a portion of a file thatholds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more portions, sub-programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this disclosure can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA, or an ASIC.

Processors or processing circuits suitable for the execution of acomputer program include, by way of example, both general and specialpurpose microprocessors, and any one or more processors of any kind ofdigital computer. Generally, a processor will receive instructions anddata from a read-only memory, or a random-access memory, or both.Elements of a computer can include a processor configured to performactions in accordance with instructions and one or more memory devicesfor storing instructions and data.

Generally, a computer will also include, or be operatively coupled toreceive data from or transfer data to, or both, one or more mass storagedevices for storing data, e.g., magnetic, magneto-optical disks, oroptical disks. However, a computer need not have such devices. Moreover,a computer can be embedded in another device, e.g., a mobile telephone,a personal digital assistant (PDA), a mobile audio or video player, agame console, a Global Positioning System (GPS) receiver, or a portablestorage device (e.g., a universal serial bus (USB) flash drive), to namejust a few.

Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

To provide for interaction with a user, implementations of the subjectmatter described in this specification can be implemented with acomputer and/or a display device, e.g., a VR/AR device, a head-mountdisplay (HMD) device, a head-up display (HUD) device, smart eyewear(e.g., glasses), a CRT (cathode-ray tube), LCD (liquid-crystal display),OLED (organic light emitting diode), or any other monitor for displayinginformation to the user and a keyboard, a pointing device, e.g., amouse, trackball, etc., or a touch screen, touch pad, etc., by which theuser can provide input to the computer.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back-endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front-endcomponent, e.g., a client computer having a graphical user interface ora Web browser through which a user can interact with an implementationof the subject matter described in this specification, or anycombination of one or more such back-end, middleware, or front-endcomponents.

The components of the system can be interconnected by any form or mediumof digital data communication, e.g., a communication network. Examplesof communication networks include a local area network (“LAN”) and awide area network (“WAN”), an inter-network (e.g., the Internet), andpeer-to-peer networks (e.g., ad hoc peer-to-peer networks).

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of any claims,but rather as descriptions of features specific to particularimplementations. Certain features that are described in thisspecification in the context of separate implementations can also beimplemented in combination in a single implementation. Conversely,various features that are described in the context of a singleimplementation can also be implemented in multiple implementationsseparately or in any suitable subcombination.

Moreover, although features can be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination can be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingcan be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

As such, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking orparallel processing can be utilized.

It is intended that the specification and embodiments be considered asexamples only. Other embodiments of the disclosure will be apparent tothose skilled in the art in view of the specification and drawings ofthe present disclosure. That is, although specific embodiments have beendescribed above in detail, the description is merely for purposes ofillustration. It should be appreciated, therefore, that many aspectsdescribed above are not intended as required or essential elementsunless explicitly stated otherwise.

Various modifications of, and equivalent acts corresponding to, thedisclosed aspects of the example embodiments, in addition to thosedescribed above, can be made by a person of ordinary skill in the art,having the benefit of the present disclosure, without departing from thespirit and scope of the disclosure defined in the following claims, thescope of which is to be accorded the broadest interpretation so as toencompass such modifications and equivalent structures.

It should be understood that “a plurality” or “multiple” as referred toherein means two or more. “And/or,” describing the associationrelationship of the associated objects, indicates that there may bethree relationships, for example, A and/or B may indicate that there arethree cases where A exists separately, A and B exist at the same time,and B exists separately. The character “/” generally indicates that thecontextual objects are in an “or” relationship.

In the present disclosure, it is to be understood that the terms“lower,” “upper,” “under” or “beneath” or “underneath,” “above,”“front,” “back,” “left,” “right,” “top,” “bottom,” “inner,” “outer,”“horizontal,” “vertical,” and other orientation or positionalrelationships are based on example orientations illustrated in thedrawings, and are merely for the convenience of the description of someembodiments, rather than indicating or implying the device or componentbeing constructed and operated in a particular orientation. Therefore,these terms are not to be construed as limiting the scope of the presentdisclosure.

Moreover, the terms “first” and “second” are used for descriptivepurposes only and are not to be construed as indicating or implying arelative importance or implicitly indicating the number of technicalfeatures indicated. Thus, elements referred to as “first” and “second”may include one or more of the features either explicitly or implicitly.In the description of the present disclosure, “a plurality” indicatestwo or more unless specifically defined otherwise.

In the present disclosure, a first element being “on” a second elementmay indicate direct contact between the first and second elements,without contact, or indirect geometrical relationship through one ormore intermediate media or layers, unless otherwise explicitly statedand defined. Similarly, a first element being “under,” “underneath” or“beneath” a second element may indicate direct contact between the firstand second elements, without contact, or indirect geometricalrelationship through one or more intermediate media or layers, unlessotherwise explicitly stated and defined.

In the description of the present disclosure, the terms “someembodiments,” “example,” or “some examples,” and the like may indicate aspecific feature described in connection with the embodiment or example,a structure, a material or feature included in at least one embodimentor example. In the present disclosure, the schematic representation ofthe above terms is not necessarily directed to the same embodiment orexample.

Moreover, the particular features, structures, materials, orcharacteristics described may be combined in a suitable manner in anyone or more embodiments or examples. In addition, various embodiments orexamples described in the specification, as well as features of variousembodiments or examples, may be combined and reorganized.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of any claims,but rather as descriptions of features specific to particularimplementations. Certain features that are described in thisspecification in the context of separate implementations can also beimplemented in combination in a single implementation. Conversely,various features that are described in the context of a singleimplementation can also be implemented in multiple implementationsseparately or in any suitable subcombinations.

Moreover, although features can be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination can be directed to asubcombination or variations of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingcan be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

As such, particular implementations of the subject matter have beendescribed.

Other implementations are within the scope of the following claims. Insome cases, the actions recited in the claims can be performed in adifferent order and still achieve desirable results. In addition, theprocesses depicted in the accompanying figures do not necessarilyrequire the particular order shown, or sequential order, to achievedesirable results. In certain implementations, multitasking or parallelprocessing can be utilized.

Some other embodiments of the present disclosure can be available tothose skilled in the art upon consideration of the specification andpractice of the various embodiments disclosed herein. The presentapplication is intended to cover any variations, uses, or adaptations ofthe present disclosure following general principles of the presentdisclosure and include the common general knowledge or conventionaltechnical means in the art without departing from the presentdisclosure. The specification and examples can be shown as illustrativeonly, and the true scope and spirit of the disclosure are indicated bythe following claims.

1. A method for multi-exposure photography, comprising: receiving ascreen touch operation; determining a collected frame corresponding tothe screen touch operation; determining, according to a first rule, asubject area image corresponding to the screen touch operation from thecollected frame corresponding to the screen touch operation;determining, according to a second rule, an alignment parameter of thesubject area image corresponding to the screen touch operation;acquiring a currently collected frame in real time; upon determiningthat the screen touch operation is the first received screen touchoperation, synthesizing, according to the alignment parameter of thesubject area image corresponding to the screen touch operation, thesubject area image corresponding to the screen touch operation into thecurrently collected frame, taking the synthesized image to be an imagewith a multi-exposure processed effect and displaying the image with themulti-exposure processed effect; or when determining that the screentouch operation is not the first received screen touch operation,synthesizing, according to corresponding alignment parameters, thesubject area image corresponding to the screen touch operation andsubject area images corresponding to all historical screen touchoperations into the currently collected frame, taking the synthesizedimage to be the image with the multi-exposure processed effect anddisplaying the image with the multi-exposure processed effect; anddetermining an image with a multi-exposure processed effect from theimages with the multi-exposure processed effect, which are generatedafter a final screen touch operation, to be a multi-exposed resultimage.
 2. The method according to claim 1, wherein the first rulecomprises: determining the subject area image corresponding to thescreen touch operation according to at least one of the screen touchoperation or an object image segmentation approach.
 3. The methodaccording to claim 2, wherein the determining the subject area imagecorresponding to the screen touch operation according to the screentouch operation and the object image segmentation approach comprises:determining an image area corresponding to the screen touch operationfrom the collected frame corresponding to the screen touch operation;inputting position information and image information contained in theimage area into a deep neural network; and recognizing the subject areaimage from the image area by means of the deep neural network.
 4. Themethod according to claim 1, wherein the second rule comprises one ofthe following: determining, according to a pre-configured alignmentparameter, the alignment parameter of the subject area imagecorresponding to the screen touch operation; conducting, according to apre-configured alignment algorithm, alignment computation on thecurrently collected frame acquired in real time and the collected framecorresponding to the screen touch operation, to obtain the alignmentparameter of the subject area image corresponding to the screen touchoperation; or conducting, according to the pre-configured alignmentalgorithm, alignment computation on the currently collected frameacquired in real time and collected frames corresponding to thehistorical screen touch operations, to obtain the alignment parameter ofthe subject area image corresponding to the screen touch operation. 5.The method according to claim 1, further comprising: receiving anediting operation initiated for the subject area image corresponding toany one or more of the historical screen touch operations; processing,according to the editing operation, the corresponding subject areaimage; taking the processed image to be an updated image with themulti-exposure processed effect; and displaying the updated image withthe multi-exposure processed effect.
 6. The method according to claim 5,wherein the editing operation at least comprises at least one of: anoperation of modifying the subject area image; or an operation ofdeleting the subject area image.
 7. An apparatus for multi-exposurephotography, comprising: a processor; and a memory, configured to storeinstructions executable by the processor, wherein the processor isconfigured to: receive a screen touch operation; determine a collectedframe corresponding to the screen touch operation; determine, accordingto a first rule, a subject area image corresponding to the screen touchoperation from the collected frame corresponding to the screen touchoperation; determine, according to a second rule, an alignment parameterof the subject area image corresponding to the screen touch operation;acquire a currently collected frame in real time; upon determining thatthe screen touch operation is the first received screen touch operation,synthesize, according to the alignment parameter of the subject areaimage corresponding to the screen touch operation, the subject areaimage corresponding to the screen touch operation into the currentlycollected frame, take the synthesized image to be an image with amulti-exposure processed effect and display the image with themulti-exposure processed effect; or when determining that the screentouch operation is not the first received screen touch operation,synthesize, according to corresponding alignment parameters, the subjectarea image corresponding to the screen touch operation and subject areaimages corresponding to all historical screen touch operations into thecurrently collected frame, take the synthesized image to be the imagewith the multi-exposure processed effect and display the image with themulti-exposure processed effect; and determine an image with amulti-exposure processed effect from the images with the multi-exposureprocessed effect, which are generated after a final screen touchoperation, to be a multi-exposed result image.
 8. The apparatusaccording to claim 7, wherein the first rule comprises: determining thesubject area image corresponding to the screen touch operation accordingto at least one of the screen touch operation or an object imagesegmentation approach.
 9. The apparatus according to claim 8, wherein inorder to determine the subject area image corresponding to the screentouch operation according to the screen touch operation and the objectimage segmentation approach, the processor is configured to: determinean image area corresponding to the screen touch operation from thecollected frame corresponding to the screen touch operation; inputposition information and image information contained in the image areainto a deep neural network; and recognize the subject area image fromthe image area by means of the deep neural network.
 10. The apparatusaccording to claim 7, wherein the second rule comprises one of:determining, according to a pre-configured alignment parameter, thealignment parameter of the subject area image corresponding to thescreen touch operation; conducting, according to a pre-configuredalignment algorithm, alignment computation on the currently collectedframe acquired in real time and the collected frame corresponding to thescreen touch operation, to obtain the alignment parameter of the subjectarea image corresponding to the screen touch operation; or conducting,according to the pre-configured alignment algorithm, alignmentcomputation on the currently collected frame acquired in real time andcollected frames corresponding to the historical screen touchoperations, to obtain the alignment parameter of the subject area imagecorresponding to the screen touch operation.
 11. The apparatus accordingto claim 7, wherein the processor is further configured to: receive anediting operation initiated for the subject area image corresponding toany one or more of the historical screen touch operations; process,according to the editing operation, the corresponding subject areaimage; take the processed image to be an updated image with themulti-exposure processed effect; and display the updated image with themulti-exposure processed effect.
 12. The apparatus according to claim11, wherein the editing operation at least comprises at least one of: anoperation of modifying the subject area image; or an operation ofdeleting the subject area image.
 13. A non-transitory computer-readablestorage medium having stored therein instructions that, when executed bya processor of a mobile terminal, cause the mobile terminal to perform amethod for multi-exposure photography, comprising: receiving a screentouch operation; determining a collected frame corresponding to thescreen touch operation; determining, according to a first rule, asubject area image corresponding to the screen touch operation from thecollected frame corresponding to the screen touch operation;determining, according to a second rule, an alignment parameter of thesubject area image corresponding to the screen touch operation;acquiring a currently collected frame in real time; when determiningthat the screen touch operation is the first received screen touchoperation, synthesizing, according to the alignment parameter of thesubject area image corresponding to the screen touch operation, thesubject area image corresponding to the screen touch operation into thecurrently collected frame, taking the synthesized image to be an imagewith a multi-exposure processed effect and displaying the image with themulti-exposure processed effect; or when determining that the screentouch operation is not the first received screen touch operation,synthesizing, according to corresponding alignment parameters, thesubject area image corresponding to the screen touch operation andsubject area images corresponding to all historical screen touchoperations into the currently collected frame, taking the synthesizedimage to be the image with the multi-exposure processed effect anddisplaying the image with the multi-exposure processed effect; anddetermining an image with a multi-exposure processed effect from theimages with the multi-exposure processed effect, which are generatedafter a final screen touch operation, to be a multi-exposed resultimage.
 14. The non-transitory computer-readable storage medium accordingto claim 13, wherein the first rule comprises: determining the subjectarea image corresponding to the screen touch operation according to atleast one of the screen touch operation or an object image segmentationapproach.
 15. The non-transitory computer-readable storage mediumaccording to claim 14, wherein determining the subject area imagecorresponding to the screen touch operation according to the screentouch operation and the object image segmentation approach comprises:determining an image area corresponding to the screen touch operationfrom the collected frame corresponding to the screen touch operation;inputting position information and image information contained in theimage area into a deep neural network; and recognizing the subject areaimage from the image area by means of the deep neural network.
 16. Thenon-transitory computer-readable storage medium according to claim 13,wherein the second rule comprises one of the following: determining,according to a pre-configured alignment parameter, the alignmentparameter of the subject area image corresponding to the screen touchoperation; conducting, according to a pre-configured alignmentalgorithm, alignment computation on the currently collected frameacquired in real time and the collected frame corresponding to thescreen touch operation, to obtain the alignment parameter of the subjectarea image corresponding to the screen touch operation; or conducting,according to the pre-configured alignment algorithm, alignmentcomputation on the currently collected frame acquired in real time andcollected frames corresponding to the historical screen touchoperations, to obtain the alignment parameter of the subject area imagecorresponding to the screen touch operation.
 17. The non-transitorycomputer-readable storage medium according to claim 13, wherein themethod further comprises: receiving an editing operation initiated forthe subject area image corresponding to any one or more of thehistorical screen touch operations; processing, according to the editingoperation, the corresponding subject area image; taking the processedimage to be an updated image with the multi-exposure processed effect;and displaying the updated image with the multi-exposure processedeffect.
 18. The non-transitory computer-readable storage mediumaccording to claim 17, wherein the editing operation at least comprisesat least one of: an operation of modifying the subject area image; or anoperation of deleting the subject area image.
 19. A mobile terminalimplementing the method according to claim 1, comprising a touch screenconfigured, wherein the mobile terminal is configured to perform animage synthesis operation in real time, and the touch screen isconfigured to display a multi-exposure synthesis effect according to thescreen touch operation.
 20. The mobile terminal according to claim 19,wherein the mobile terminal has a real-time preview function with themulti-exposure synthesis effect.